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ADDRESSING  HUMAN  FACTORS  GAPS  IN  CYBER  DEFENSE 


Panel  Chair: 

Alex  Vieane,  Colorado  State  University 

Panelists: 

Dr.  Gregory  Funke,  Air  Force  Research  Laboratory 
Dr.  Robert  Gutzwiller,  Space  and  Naval  Warfare  Systems  Center  Pacific 
Dr.  Vincent  Mancuso.  MIT  Lincoln  Laboratory 
Dr.  Ben  Sawyer,  MIT  AgeLab 
Dr.  Christopher  Wickens,  Colorado  State  University 

Cyber  security  is  a  high-ranking  national  priority  that  is  only  likely  to  grow  as  we  become  more  dependent 
on  cyber  systems.  From  a  research  perspective,  currently  available  work  often  focuses  solely  on 
technological  aspects  of  cyber,  acknowledging  the  human  in  passing,  if  at  all.  In  recent  years,  the  Human 
Factors  community  has  begun  to  address  human-centered  issues  in  cyber  operations,  but  in  comparison  to 
technological  communities,  we  have  only  begun  to  scratch  the  surface.  Even  with  publications  on  cyber 
human  factors  gaining  momentum,  there  still  exists  a  major  gap  in  the  field  between  understanding  of  the 
domain  and  currently  available  research  meant  to  address  relevant  issues.  The  purpose  for  this  panel  is  to 
continue  to  expand  the  role  of  human  factors  in  cyber  research  by  introducing  the  community  to  current 
work  being  done,  and  to  facilitate  collaborations  to  drive  future  research.  We  have  assembled  a  panel  of 
scientists  across  multiple  specializations  in  the  human  factors  community  to  have  an  open  discussion 
regarding  how  to  leverage  previous  human  factors  research  and  current  work  in  cyber  operations  to 
continue  to  push  the  bounds  of  the  field. 


Defense  in  cyberspace  is  a  high-ranking  national  security 
concern,  supported  by  recent  congressional  testimony  noting 
that  the  U.S.  saw  a  782%  increase  (from  5,503  to  48,562 
cases)  in  the  number  of  reported  cyber  attacks  against  federal 
agencies  from  2006  to  2012  (GAO-13-462T).  Recent 
revelations  of  large  scale  data  breaches  in  both  the  private  and 
public  sectors  (e.g.,  Kuranda,  2015)  further  emphasize  the 
need  for  improved  cyber  defense.  In  response  to  these  and 
other  emerging  cyber  threats,  the  President’s  Fiscal  Year  2017 
budget  called  for  $19  billion  in  cyber  security  spending,  an 
approximate  35  percent  increase  over  current  spending  levels 
(Calmes,  2016).  In  addition,  for  the  first  time  the  budget 
explicitly  identifies  human  factors  research  in  cyber  as  a 
priority  (Waldrop,  2016). 

Given  the  importance  of  cyber  defense  to  national 
security,  it  is  critical  that  the  human  factors  research 
community  continues  to  contribute  to  this  difficult  problem 
area.  Significantly,  three  panel  discussions  devoted  to  human 
factors  issues  in  cyber  have  been  featured  in  recent  Annual 
Meetings  of  the  Human  Factors  and  Ergonomics  Society  (i.e., 
Knott  et  ah,  2013;  Mancuso  et  ah,  2014;  McNeese  et  ah, 
2012).  These  panels  brought  together  researchers  with  a 
variety  of  backgrounds  to  discuss  their  unique  perspectives  on 
the  role  of  human  factors  in  cyber  operations,  and  have  helped 
initiate  discussions  concerning  contributions  human  factors 
researchers  and  practitioners  can  make  to  cyber  security.  The 
purpose  of  the  current  panel  is  to  continue  this  trend. 


PANELIST  ABSTRACTS 
Training  Challenges  in  Cyber  Security 

Dr.  Gregory  Funke 
Air  Force  Research  Laboratory 

With  our  nation’s  growing  reliance  on  cyber  systems,  the 
need  for  resilient  and  effective  cyber  operations  has  become 
increasingly  apparent  (Maybury,  2015).  To  date,  most  of  the 
research  in  cyber  security  has  been  focused  on  technology 
applications  (e.g.,  network  intrusion  detection  sensors  and 
algorithms,  intelligent  system  designs,  etc.),  thus  ignoring 
consideration  of  the  critical  roles  humans  play  (e.g.,  applying 
these  technologies,  assimilating  information,  and  arriving  at 
threat  diagnoses)  in  cyber  operations.  To  fully  capitalize  on 
the  promises  of  advanced  cyber  security  technologies  we  must 
also  understand  the  roles,  tasks,  and  responsibilities  of  human 
operators  in  these  environments. 

At  present  training  for  careers  in  cyber  defense  are  largely 
unstandardized.  Novice  cyber  defenders  are  likely  to  have 
disparate  backgrounds,  leading  to  partial  or  incomplete 
understanding  of  system  vulnerabilities.  These  deficits  are 
currently  overcome  through  on-the-job-training  and  certificate 
programs,  but  this  requires  commitment  of  significant 
additional  investments  (in  the  forms  of  time  and  resources) 
transforming  novices  into  expert  cyber  defenders.  Even  after 
defenders  develop  a  degree  of  expertise,  the  rapid  pace  of 
advancements  in  malicious  software  and  defensive  counter 
software  frequently  outpaces  their  opportunities  to  remain 
current. 

In  military  contexts,  this  problem  is  exacerbated  by  job 
stressors,  such  as  long  work  shifts,  significant  manpower 
shortages,  and  slow  moving  government  bureaucracy  which 
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impairs  defenders’  choices  with  regard  to  software  platforms 
(Chappelle  et  al.,  2013).  The  net  result  appears  to  be  that  many 
cyber  defenders  receive  substantial  training  in  the  military,  but 
separate  as  soon  as  they  are  able,  transitioning  to  higher  paid, 
and  potentially  less  stressful,  work  in  the  private  sector, 
further  exacerbating  manpower  shortages  in  many  units  (Apps 
&  Goh,  2013). 

The  human  factors  community  is  in  a  unique  position  to 
make  contributions  in  this  area,  as  many  of  the  relevant  issues 
have  been  successfully  addressed  in  existing  work  contexts 
(e.g.,  shift  work),  while  others,  such  as  approaches  to 
accelerate  training,  provide  novel  areas  for  researchers  to 
focus  their  efforts  on. 

Cyber-Cognitive  Situation  Awareness 
Dr.  Robert  Gutzwiller 

Space  and  Naval  Warfare  Systems  Center  Pacific 

Cyberspace  is  a  realm  of  dynamic  information  transmittal. 
Information  is  literally  moving  at  the  speed  of  light  through 
hundreds  of  thousands  of  connections  over  a  vast  array  of 
networks  and  billions  of  devices.  Naturally  this  precludes  an 
easily  conjured  understanding  of  what  any  given  cyber  analyst 
sees  and  does.  For  defenders  in  particular,  I  take  some  effort  to 
define  that  cyberspace  defense  is  multifaceted.  Defense 
improvements  can  be  as  simple  as  network  users  choosing  not 
to  click  on  phishing  emails;  given  the  proclivity  for  these 
types  of  cyber  vulnerabilities  this  is  a  relevant  area  to  address. 

But  for  the  current  purposes  I  believe  we  must  dig  deeper  into 
another  human  element  in  cyber  defense,  that  of  the  analyst 
(e.g.,  D'Amico  et  al.,  2005).  Analysts  operate  closely  with  the 
literal  network  communication  and  transmission,  using 
software  tools  to  examine  down  to  the  level  of  packets  of 
information  and  internet  protocol  addresses.  The  tools,  often 
command-line  driven,  monitor  the  network  activity,  help  parse 
and  search  through  information,  and  track  potential  and 
current  threats  to  security.  These  threats  change  on  a  daily 
basis  with  every  new  patch  and  update  to  programs  and 
operating  systems,  as  well  as  hardware  changes.  The  number 
of  threats  is  always  increasing,  and  there  are  an  unknown 
amount  of  so-called  "zero-day"  threats  which  have  no  current 
mitigation.  In  all  respects,  defenders  are  at  the  mercy  of  cyber 
attackers. 

What  I  am  attempting  to  convey  is  how  necessary  it  is  for 
cyber  defenders  to  perceive  and  understand  disparate  elements 
of  network  information  in  order  to  determine  whether  a 
malicious  entity  or  program  is  present  or  attempting  an  attack. 
Currently  this  information  is  noisy,  it  is  rarely  correlated  and  it 
is  almost  never  linked  with  the  users’  goal  of  maintaining 
mission-critical  systems  or  projecting  the  ability  to  execute 
future  courses  of  action.  Cyber  defense  is  easily  related  to  a 
theory  of  situation  awareness  (Endsley,  1995),  and  thus  stands 
a  good  chance  of  benefiting  from  similar  study.  It  should  be 
noted  that  cyber  situation  awareness  as  a  concept  is  actually 
nothing  new:  Tim  Bass  coined  the  phrase  over  15  years  ago 
(Bass,  2000).  He  was  keen  at  the  onset  to  point  out  the 
technological  elements  to  cyber  awareness  -  ways  that  the 
system  could  be  made  to  identify,  share  and  fuse  information, 
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to  enhance  a  computer's  representation  of  the  environment. 

But  it  was  simultaneously  emphasized  that  this  situation 
awareness  was  a  critical  necessity  for  humans  to  possess.  It 
was  not  enough  in  this  definition  to  fuse  information  and 
represent  it  within  the  system,  and  then  assume  human 
awareness. 

Unfortunately  the  audience  for  Bass's  article  appears  to 
have  fixated  on  the  technocentric  bent  of  cyber  situation 
awareness  and  its  various  difficulties.  The  human,  as  so  often 
is  the  case,  has  in  turn  been  neglected  in  cyber  defense,  only 
recently  returning  to  focus  (e.g.,  Champion  et  al.,  2012; 
Mancuso  et  al.,  2012;  Giacobe,  2013;  Gutzwiller  et  al.,  2015). 
I  promote  cyber-cognitive  situation  awareness  as  the  proper 
terminology  to  identify  that  in  this  domain,  we  are  interested 
in  the  human  perception,  understanding,  and  prediction  of  the 
cyber  defensive  space.  Naturally,  this  is  a  human-systems 
integration  perspective,  and  one  that  fits  seamlessly  with  that 
of  cognitive  engineering  efforts  that  are  just  beginning  in  the 
cyber  domain.  It  is  critical  that  the  community  developing 
interfaces  and  visualizations  for  cyberspace  recognize  that 
awareness  is  not  achieved  by  simply  displaying  all  of  the 
possible  information  from  the  system;  instead  we  need 
situated  information,  to  incorporate  the  needs  of  the  operators, 
and  a  system  which  can  account  for  the  dynamics  of  both. 

Cognitive  Coordination  of  Multi-Sensor  Cyber  Data 

Dr.  Vincent  Mancuso 
MIT  Lincoln  Laboratory 

In  Joint  Publication  3-13,  building  and  maintaining 
Cyberspace  Situational  Awareness  (Cyber  SA)  is  identified  as 
a  key  function  for  cyber  operations.  From  a  computer  science 
and  engineering  perspective.  Cyber  SA  is  an  issue  solved 
through  maturing,  improving,  and  expanding  cyber  sensor 
technologies.  The  goal  of  such  technologies  is  to  provide  data 
from  across  a  cyber  landscape  to  create  a  more  complete 
picture  of  the  current  situation  and  potential  emerging  threats. 
While  these  technologies  provide  a  greater  breadth  of  data  to 
an  analyst,  the  challenge  of  triangulating  and  fusing  the  data 
across  sensors  becomes  unmanageable  as  the  amount  of 
available  data  increases.  Unlike  traditional  operating 
environments,  in  cyber  there  is  a  fundamental  disconnect 
between  the  analyst  and  their  environment,  and  cyber  sensor 
data  is  devoid  of  ecological  and  contextual  anchors  (McNeese, 
Mancuso,  McNeese,  Endsley,  &  Forster,  2013).  Without 
anchors,  the  relationships  across  cyber  sensors  and  data  are 
only  visible  through  an  interpretation  of  data  within  the 
cognitive  mind  of  an  individual. 

Currently,  this  task,  also  known  as  multi-sensor 
information  fusion,  is  done  manually,  often  through  multiple 
pieces  of  software  with  little  or  no  interoperability.  In  order  to 
gain  a  more  holistic  understanding  of  the  event,  operators 
must  extract  data  from  a  tool  onto  a  distributed  cognitive 
artifact,  such  as  a  word  document  or  notebook  paper,  and 
manually  “pivot”  to  another  piece  of  software  and  search  for 
related  data.  This  pivot  is  done  using  correlational  linkages, 
often  in  the  form  of  text-based  data  that  is  made  up  of  time- 
stamps,  IP  addresses,  host  names,  port  numbers,  and  codes. 
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Because  of  this,  the  relationships  and  interdependencies  across 
sensors  exist  only  in  the  mind  of  an  individual  analyst,  taxing 
their  working  memory  and  making  the  maintenance  of  their 
Cyber  SA  challenging.  This  is  even  further  magnified  when 
scaling  to  the  team,  or  organizational  level,  as  many  of  the 
cyber  analysts  have  diverging  mental  models  of  the 
environment,  divided  across  multiple  types  of  cyber 
operations  (Tyworth,  Giacobe,  Mancuso,  McNeese,  &  Hall, 
2013).  Given  the  importance  of  Cyber  SA,  and  the  importance 
of  the  human,  there  is  a  critical  need  for  Human  Factors 
research  to  compliment  the  emerging  sensors  and  data  from 
other  research  domains. 

A  potential  solution  to  this  problem  may  be  found  in 
interactive  visual  analytic  techniques,  such  as  coordinated 
views.  These  techniques  help  users  perceive  complex  and 
dynamic  relationships  in  the  underlying  data,  facilitating 
comprehension  and  pattern  recognition.  Coordinated  views 
have  been  used  in  multiple  domains  (cf.  Roberts,  2007), 
however  current  work  in  cyber  is  fairly  limited  and  does  not 
account  for  the  human  (e.g.  Noel,  Jacobs,  Kalapa,  &  Jajodia, 
2005).  In  pursuit  of  this  problem,  we  present  the  Integrated 
Open  Source  Architecture  (IOSA),  as  a  platform  to  facilitate 
the  coordination  of  cyber  analysis  across  numerous  sensor 
platforms  and  analysis  tools.  The  goal  of  IOSA  is  to  automate 
the  manual  “pivoting”  process  that  cyber  analysts  are  forced  to 
do  in  current  operations.  IOSA  is  built  as  a  wrapper  for 
numerous  cyber  tools,  but  rather  than  having  to  manually 
extract  and  fuse  data  across  platforms,  an  operator  can  quickly 
jump  between  platforms.  This  coordination  not  only  will 
support  the  analysis  process  by  shortening  the  cognitive 
overhead  of  cyber  data  fusion,  but  will  also  help  reduce  the 
difficulty  and  time  associated  with  other  articulation  tasks 
such  as  briefing  and  report  generation.  In  this  talk,  we  will 
discuss  these  human  factors  issues,  and  provide  examples  of 
solutions  that  better  enable  the  development  and  maintenance 
of  Cyber  SA. 

Acknowledgements.  This  material  is  based  upon  work 
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and/or  FA8702-15-D-0001.  Any  opinions,  findings, 
conclusions  or  recommendations  expressed  in  this  material  are 
those  of  the  author(s)  and  do  not  necessarily  reflect  the  views 
of  the  U.S.  Air  Force. 

Vigilance  in  Cyber  Security 

Dr.  Ben  Sawyer 
MIT  Age  Lab 

Cyber-defenders  have  been  shown  vulnerable  to  the 
vigilance  decrement  (Sawyer  et  al.,  in  press),  a  well-studied 
weakness  in  human  cognition.  Characterized  by  repetitive, 
seemingly  simple  tasks  that  escalate  over  time  into  hard, 
stressful  work  and  compromised  operator  performance 
(Warm,  Parasuraman,  &  Matthews,  2008),  vigilance  has  been 
previously  investigated  in  contexts  including  air  traffic  control 
and  medical  monitoring.  In  the  cyber  context,  where  display 
information  density  is  several  orders  of  magnitude  above  that 
seen  in  the  aforementioned  domains,  the  need  to  keep  operator 


workload  from  exceeding  the  information  processing 
capacities  of  security  operators  is  especially  crucial.  Indeed, 
the  potential  use  of  vigilance-promoting  conditions  as  an 
attack  vector  has  been  suggested  (Sawyer  et  al.,  in  press),  and 
recent  research  into  prevalence  effects  in  email-delivered 
cyberattacks  shows  such  efforts  may  be  efficacious  (Sawyer, 
2015).  A  prevalence  denial  attack  (PDA)  would  flood  a 
network  with  “grey  signals,”  purpose-built  to  be  flagged  by 
algorithmic  defense  systems  for  human  inspection,  yet  easily 
identified  as  non-threats  by  said  operators  (see  Sawyer  et  al., 
in  press).  This  denial  &  deception  (D&D)  tactic  would 
artificially  depress  the  signal  probability  of  candidate  events 
presented  to  cyber-defenders,  resulting  in  compromised  cyber¬ 
defender  accuracy  and  allowing  genuine  attacks  a  greater 
chance  to  avoid  human  detection.  PDA  D&D  cyber  tactics 
clearly  expose  the  importance  of  the  human  factor  in  cyber¬ 
warfare,  as  well  as  the  danger  of  network  defense  strategy 
overly  focused  on  algorithmic  protections.  Such  foreshadows 
a  new  class  of  cyber-cognitive  exploits  that  will  effectively 
“hack  the  human”  rather  than  the  machine. 

Attention  Switching  in  Cyber  Security 

Dr.  Christopher  Wickens 
Colorado  State  University 

Cyber-security  analysts  at  whatever  level  they  serve,  must 
engage  in  extensive  attention  switching.  The  cyber  security 
task  is  far  more  complex  than  one  of  simple  single  task 
vigilance.  In  cyber-security,  such  switching  may  be  between 
different  threads  within  a  heterogeneous  message  stream;  it 
may  involve  switching  between  different  hypotheses  regarding 
the  sender’s  intent  within  a  single  thread;  it  may  involve 
physical  pivoting  between  different  screens,  or  it  may  involve 
switching  attention  to  deal  with  an  interruption  and  perform  a 
very  different  task. 

At  the  same  time  that  attention  switching  at  many  levels 
(cognitive,  physical,  task)  is  necessary,  ample  research  also 
indicates  that  it  is  costly  (see  Wickens,  Gutzwiller,  & 
Santamaria,  2015  for  a  review).  Too  much,  or  too  rapid 
switching  can  destroy  task  continuity  (even  as  too  few  and  too 
slow  switching  can  create  unwanted  cognitive  tunneling).  The 
purpose  of  this  presentation  will  be  to:  a)  identify  and  analyze 
in  more  detail,  different  forms  of  switching,  and  their  costs 
and  benefits  within  a  particular  cyber  security  context;  b) 
apply  a  recent  model  of  task  switching  -  the  STOM  model 
(strategic  task  overload  management,  Wickens,  Gutzwiller,  & 
Santamaria,  2015)  -  to  switching  and  multi-tasking  scenarios 
in  the  cyber- security  analyst’s  workplace;  and  c)  employ  this 
model  to  derive  certain  mitigation  strategies,  in  training, 
procedures,  displays,  and  workplace  layout,  that  may  assist  the 
analysts  in  their  task. 
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